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ABSTRACT 



Drawing on a previous study exploring expertise development 
in management (J. Arts, W. Gijselaers, and H. Boshuizen, 2000), this study 
investigated how dimensions of mood may affect recall in studies on expertise 
development and whether sanple composition (distribution of males and 
females) may account for possible differences in the recall function. A data 
sample taken from the previous study* s data set contained 18 novices, 14 
students at the end of the first year of the management program, 22 students 
at the end of the second year, 15 from the third year, and 16 fourth-year 
students shortly before graduation. The materials were two cases on 
organizational development. Results indicate the possibility that gender and 
mood may affect recall in studies on expertise development. It was also shown 
that subjects with the same level of expertise may not be treated as 
replicates. Some of the error variance may be explained by the systematic 
bias sources of gender and mood. An appendix contains the instrument used in 
the study and a description of the case studies. (Contains 17 references.) 
(SLD) 
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Introduction 



Studies in the domain of medicine have shown that expertise development is not 
simply a process of extending knowledge in existing semantic networks (Schmidt, 
Norman & Boshuizen, 1990; Patel & Groen, 1991; Schmidt & Boshuizen, 1993). In 
fact, size of semantic networks is not linearly related to expertise development. But, 
the relationship between expertise and features of semantic networks is far more 
complex. It appears that one of the most consistent findings in novice-expert 
comparisons conducted in the area of medicine, is the so-called intermediate effect. 
Expert physicians typically recall fewer propositions than intermediate medical 
students who recall more than novices (Schmidt & Boshuizen, 1993). This 
phenomenon is defined as the so-called intermediate effect in clinical case 
representation studies. 

Although the intermediate effect in medicine, and other domains of expertise, appears 
to be very consistent, recent studies by Vaatstra (1996) and van de Wiel (1997) failed 
to replicate the earlier findings of Schmidt and Boshuizen (1993). In both studies, 
experts produced significantly more propositions than the student groups (novices and 
intermediates). For example. Van de Wiel (1997) found in her study a monotone 
increasing recall function with expertise for four cases in internal medicine. Vaatstra 
(1996) found a similar phenomenon in the recall function in a study on expertise 
development in accounting. In both studies it was hypothesized that the failure to 
replicate the intermediate effect may be explained by case or task difficulty. 
Basically, these researchers hypothesized that the cases used required elaborate 
processing of the data even by experts. In turn, the elaborate data processing would 
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inhibit the use of compiled or encapsulated knowledge, which is usually observed 
when experts analyze a case. Confirmation for this hypothesis may be found in the 
cases used by Vaatstra (1996). Her study included lengthy and detailed financial 
information about managing organizations. Analysis of financial data requires in 
depth understanding of financial figures to analyze the cases. In addition calculation 
of new financial indexes is required to arrive at a conclusion. Vaatstra (1996) found 
differences in application of knowledge, but she didn’t find differences in literal recall 
of the information presented. She concluded that it was not possible to explain from 
her data why the intermediate effect didn’t show. Van de Wiel (1997) tried to 
develop several alternate hypotheses, next to the task difficulty hypothesis, for the 
absence of the intermediate effect. But her material didn’t allow examining alternate 
hypotheses. Her final conclusion was that “However, it remains unclear why the 
experts in the present study elaborately processed the cases” (Van de Wiel 1997, p. 
105). 

Despite the number and variety of studies demonstrating the intermediate-effect and 
the consistent patterns of change that have been found in problem-solving as expertise 
increases, some indications exist that other factors may as well influence literal recall. 
For example, Balch and Myers (1999) show that in memorizing words, subjects’ 
recall of words may be affected by changes in dimensions of mood (pleasantness and 
arousal). They assume that mood-item connections are learned between mood and 
items to be remembered. Recalling items under different mood conditions than 
learned may then account for reduced recall. An additional factor may be found in the 
motivation of subjects to analyze the cases given during the experiments. Mayer 
(1999) points out that in research on development of problem-solving skill, the role of 
motivational aspects has largely been ignored. According to Mayer (1999), theories 
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on cognitive motivation predict that learning and development of problem-solving 
skills depend partly whether students think harder and process the material more 
deeply when they are interested rather than uninterested. 

From a methodological and statistical perspective it is quite surprising to notice that 
in many studies on expertise development, unlike in other areas of educational 
research, variability within conditions as additional explanatory factor in recall results 
has never been further examined. It is generally assumed that subjects within a 
condition may be considered as replicates. Within condition variability may then be 
interpreted as measurement of error. This is a distinctly different situation from 
educational research areas addressing reliability and validity issues of high inferential 
measures like students’ evaluations of university teaching. In this particular domain, 
many studies show how motivation and background characteristics of subjects may 
affect ratings of teaching quality (Marsh, 1984). 

The present study attempts to examine whether certain background variables affect 
the results in recall functions in expertise development. This study draws further 
upon the Arts, Gijselaers and Boshuizen (2000a) study that explores expertise 
development in the area of management. Design conditions of that particular study 
made it possible to conduct some exploratory analyses on whether within condition 
variability may be explained by certain subject characteristics. In particular attention 
was paid to how dimensions of mood may affect recall in studies on expertise 
development and whether sample composition (distribution of males and females) 
may account for possible differences in the recall function. The present paper 
examines whether subject-dependent variables (in case mood and gender) affect 
recall, aims at analyzing within condition variability to find sources of subject bias. 
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Method 



Background of the present study. The present study draws further upon a research 
project by Arts, Gijselaers and Boshuizen (2000a) aiming at development of expertise 
in management sciences. Due to the design of this project it was possible to conduct 
additional analyses focusing on potential bias sources due to subjects’ background 
characteristics. In particular we were interested in effects of mood on recall and on 
gender effects on recall. 

Research questions. 

The first question addressed the issue whether literal recall in expertise studies may be 
biased by sex differences. Vispoel and Forte Fast (2000) state that “The 
preponderance of research evidence suggests that sex differences in many educational 
and psychological domains are actually quite small, have been declining over time, 
and are more situationally and culturally dependent than sex linked. However, sex 
differences in some areas still persist.” For example, Vispoel and Forte Fast (2000) 
report that sex differences in self-concept frequently emerge in stereotypical patterns 
in which men score higher than women in math, problem-solving, whereas women 
score higher in verbal and artistic domains. These differences appear to be reasonably 
consistent across age levels. It has also been found that males score higher on 
standard measures of self-esteem than females, although the differences appear to be 
small (Kling, Hyde, Showers & Buswell, 1999). Finally, some research on drop-out 
and success rates within undergraduate college suggests that women have higher 
success rates on exams, less drop-out and are more intrinsically motivated (Gijselaers, 
1997; Gijselaers & Ramaekers, 1990). The first research question in the present study 
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address the issue whether sex differences may account for within condition 
variability. 

The second question concerned the potential effects of mood on recall. Recent 
studies on mood-dependent memory show that participants’ recall is influenced by 
mood changes (Rich, Macaulay & Ryan, 1994; Balch, Myers & Papotto, 1999). A 
typical design for this type of study is that participants learn a set of words under one 
mood condition, and have to recall them under a different or the same mood 
condition. There is evidence to suggest that under the changed mood condition, word 
recall seems to be mediated through mood changes. It is hypothesized that mood 
changes may be considered as a context-dependent memory effect. That is, if context 
changes, participants’ recall is reduced. This finding fits in theories about effects of 
context specificity on memory retrieval stating that memory retrieval is partly 
dependent on environmental cues during encoding information (e.g. Regehr & 
Norman, 1996). 

The present study examines whether mood by itself affects participants’ recall. In this 
respect our study differs from traditional mood studies that examine the effect of 
mood changes. In this particular study we were interested in the question whether 
mood during the experiment influences participants’ readiness to process a case, 
which in turn may influence storage and retrieval of information. 

Since one of the scopes of the present study was to examine gender effects, a data 
sample was taken from the Arts, Gijselaers and Boshuizen (2000) project’s data set. 
This sample excludes the different levels of post-graduate expertise. Note, the post- 
graduate samples contained for the most part only male subjects. We asked 18 
students entering the program (novices), 14 students end of first year program, 22 



Gijselaers, W.H., & Arts, J.A. Effects of Mood and Gender 
Paper presented at AERA, April 2000, New Orleans 



5 



students (end of second year), 15 students (end of third year), and 16 (fourth year 
university students, a few months before graduation) in the school of management. 
Subjects received a small compensation for their participation. Table 1 contains the 
sample composition. 



Table 1: Distribution of Subjects Between Different Conditions 





Novices 


End of year 


Data Sample 
End of 2"'' yr 


End of 3'^'* yr 


End of yr 


Total 


Male 


14 


10 


18 


7 


12 


61 


Female 


4 


4 


4 


8 


4 


24 


Total 


18 


14 


22 


15 


16 


85 



Procedure. A standard experimental procedure for expert-novice studies was 
followed (Patel & Groen, 1991). Subjects were informed that they would get several 
assignments after they got a text for a limited period of time. Reading time was 
restricted to .4 seconds per word. In total subjects were allowed 2.15 minutes for 
Case A, and 2.50 minutes for Case B. The first assignment was that they were asked 
to write down as much as possible as they could remember. The second assignment 
asked them to analyze and diagnose the case. Each subject was shown a written 
description of a management case on organizational development. 

Materials. The materials consisted of two cases on organizational development. 
Case A consisted - in the Dutch version - of 339 words. Case B contained - in the 
Dutch version - 426 words. A translated version of these cases is presented in the 
Appendix section. The cases were considered to be at the level of intermediate 
difficulty (end of 4'*’ year). Both cases were similar with respect to format and 
procedure applied during the experiment. These cases were specifically designed for 
research purposes. Two experts from professional world and two university teachers 
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worked together with the researchers to develop cases reflecting problem situations 
graduates may encounter in real-world. Cases were designed to contain a certain 
amount of case-relevant cues and case-irrelevant cues. A canonical model was 
developed in advance by this group to identify the number of propositions, relevant 
and irrelevant propositions, and correct and incorrect inferences and diagnoses. 

After offering the first case, subjects were asked to fill in the PANAS Scale, 
measuring positive and negative affect (Watson, Clark & Tellegen, 1988). PANAS is 
a validated mood scale, consisting of twenty items measuring positive mood 
(pleasantness, arousal, etc.) and negative mood (irritable, ashamed, etc.). Subjects 
were asked to indicate on a five point scale (1= very slightly, 5= extremely) to what 
extent the feelings described were applicable to them over the past two weeks. After 
the second case, subjects received the PANAS Scale again. But now we were asking 
how they felt at the present moment. 

Analysis. The case texts were segmented in propositions. Propositions consisted of 
two concepts connected by a qualifier (Kintsch, 1974). Both recall and case analysis 
protocols were scored against the original case text propositions to compose the recall 
function. Recall protocols were scored on the count of literal reproductions of the 
text. Propositions in the recall protocols were classified as case-relevant or case- 
irrelevant, based on experts' classification. 

Results and Discussion 

Free recall. Figure 1 shows the number of correct propositions recalled by the 
subjects in the free-recall protocols. Both the results of Case A and Case B are 
included in figure 1. Results from a two-way Simple Factorial ANOVA (dependent = 
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amount of recall, factors are level of expertise and gender; Method = Unique) are 
presented in table 2 (Case A) and table 3 (Case B). 



Number of Propositions 



— Case B Male 


- -O- ■ Case B Female 


^ Case A Male 


" & ■ Case A Female 




Level of Expertise 

Figure 1: Number of correct propositions recalled 



Results show that Male students recalled in Case B [F(l,75) = 3.15, MSe = 23.7, p < 
.08] and Case A [F(l,75) = 9,87 MSe = 15,5, p < .002] significantly fewer information 
than female students. However, overall differences between level of expertise were 
only statistically significant for Case A [F(4,75) = 3.6 MSe = 15.5, p < .009] and not 
significant for Case B [F(4,75) =1.1, MSe = 23.7, p < .36]. No significant two-way 
interaction effects were found. 



Table 2: AN OVA table for Case A 



Source Type III Sum of 
Squares 


df 


Mean Square 


F 


Sig. 


Corrected Model 


499.634 


9 


55.515 


3.573 


.001 


Intercept 


4900.653 


1 


4900.653 


315.441 


.000 


Level of Expertise 


224.797 


4 


56.199 


3.617 


.009 


Gender 


153.342 


1 


153.342 


9.870 


.002 


Expertise * Gender 


60.308 


4 


15.077 


.970 


.429 


Error 


1165.190 


75 


15.536 






Total 


7266.000 


85 








Corrected Total 


1664.824 


84 
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Table 3: ANOVA table for Case B 



Source 


Type III Sum 
of Squares 


df 


Mean Square 


F 


Sig. 


Corrected Model 


259.567 


9 


28.841 


1.217 


.298 


Intercept 


3869.378 


1 


3869.378 


163.212 


.000 


Level of Expertise 


104.114 


4 


26.029 


1.098 


.364 


Gender 


74.670 


1 


74.670 


3.150 


.080 


Expertise * Gender 


17.623 


4 


4.406 


.186 


.945 


Error 


1778.080 


75 


23.708 






Total 


6560.000 


85 








Corrected Total 


2037.647 


84 









This implies that female subjects recall significantly more correct information than 
male subjects. The results in figure 1 suggest that the largest differences between 
male and female subjects occur at the intermediate level. 

Recall during problem-solving. A statistically significant main effect of gender was 
found for Case A [F(l,69) = 6.01, MS^ = 8.17, p < .017]. In case A female subjects 
recalled more propositions from text during solving the management case. No 
significant gender differences were found for case B [F(l,65) = 2.66, MSe = 6.62, p < 
.10]. In addition it was found that no differences occur between male and female 
subjects concerning diagnostic accuracy or amount of inferences. 

Relevant vs. irrelevant recall in recall condition. In-depth analyses of the recall in the 
recall condition showed that gender differences occur primarily upon the produced 
amount of literal correct recall that is relevant to the case. For both cases we found 
non significant differences for irrelevant recall whereas significant differences were 
found for relevant recall. 

Mood. 

Two dominant effects consistently emerged in the study on the effects of mood on 
recall. Moderate correlations were found between measures of positive mood (over 
the past two weeks) and recall in Case A, whereas negative mood (over the past two 
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weeks) correlated with the number of inferences during the recall assignment of Case 
A. In addition positive mood at the present moment was positively correlated with 
inferences in Case A, and nervousness at the present moment was negatively 
correlated with recall. For Case B substantially less correlations were found for 
measures of mood and recall measures. 



Table 4: Correlations between Mood, Recall and Inferences 





Case A 




Case B 




Mood 


Recall 


Inferences 


Recall 


Inferences 


Present Moment 

Nervous 

Strong 


-.29* 


.37* 






Past Two Weeks 
Active 
Determined 
Jittery 


.24* 


.24* 


.243* 




Afraid 




.26* 






Upset 




.25* 






Irritable 




.46* 






Strong 

Attentive 




.25* 




.26* 


Positive Mood 
Negative Mood 


.25* 


.32* 







N = 71 Subjects, Listwise Deletion 



To clarify whether Mood measures were correlated with confounding variables like 
gender and level of expertise, additional regression analyses were conducted with 
Number of Inferences Case A as dependent variable, and Measures of Mood and 
Level of Expertise as independent variables. Tables 5 and 6 provide a summary of 
this analysis. These tables show that the Negative Mood measure still explains 
variance of the dependent variable after entering level of expertise. That is, level of 
expertise explains the largest amount of variance, but Negative Mood (over the past 
two weeks) explains an additional amount of 3.2% variance. 
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Table 5: Model Summary 



R R Square Adjusted R 
Square 


Std. Error of the 
Estimate 


Change 

Statistics 






Model 




R Square F Change df 1 


df2 Sig. F Change 






Change 






1 ,699 ,489 ,482 


1,7432 


,489 66,013 1 


69 


,000 


2 ,722 ,521 ,507 


1,7002 


,032 4,541 1 


68 


,037 


a Predictors: (Constant), Level of Expertise 








b Predictors: (Constant), Level of Expertise, Negative Mood Past Two Weeks 







Table 6: Coefficients 







Unstandardized 




Standardized 


t 


Sig. 


Collinearity 








Coefficients 




Coefficients 






Statistics 




Model 




B Std. 


Error 


Beta 






Tolerance 


VIF 


1 


(Constant) 


2.006 


.350 




5.737 


.000 








Level of 


1.204 


.148 


.699 


8.125 


.000 


1.000 


1.000 


2 


Expertise 

(Constant) 


.656 


.719 




.913 


.365 








Level of 
Expertise 


1.150 


.147 


.668 


7.838 


.000 


.970 


1.031 




Negative 
Mood Past 


.845 


.397 


.182 


2.131 


.037 


.970 


1.031 




Two Weeks 

















a Dependent Variable: Number of Inferences Case A 



Conclusions 

The results of the current study are limited to the task environment (Cases A & B), 
measures we have used (PANAS) and by the properties of the data sample. Despite 
these limitations, the present study shows the possibility that gender and mood may 
affect recall results in studies on expertise development. It is possible that recall 
results may differ depending on sample composition (male - female distribution) and 
subjects’ background characteristics like mood. It is also shown in this study that 
subjects with the same level of expertise may not be treated as replicates, or that 
within level of expertise variability is largely due to random error. It looks like some 
of the error variance may be explained by systematic bias sources (gender and / or 
mood). 
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Although in the present study it was never planned in advance to examine bias 
sources, and therefore the statistical power or the research design was not optimal, it 
is important to understand how subject characteristics may influence outcome 
measures like recall. In summary, the present study confirms that recall and amount 
of inferences (during recall) are related to level of expertise. This result is in line with 
the majority of research on expertise development. But in addition to the explanatory 
value of level of expertise, measures of mood and gender are also associated with 
recall functions. Both variables seem to add on the effects of level of expertise. The 
gender effect seems to be strongest for the intermediate group. That is, female 
students produce substantially more propositions than male students in the 
intermediate condition. At the same time, gender effects were not found for measures 
describing the process of case analysis (number of correct diagnoses, number of 
inferences). This outcome may suggest that female students attempt to be more 
accurate while writing down what they remembered in the recall assignment and 
analysis assignment. Whether this may be interpreted as a social desirability effect, or 
motivational effect is unclear (given the available data). This outcome suggests at 
least that sample composition (female - male distribution) may account for variability 
within levels of expertise. Since the intermediate effect is always found for subjects 
just before graduation, the question arises whether the failure to replicate the 
intermediate effect in some studies may not only be attributed by task difficulty - as 
hypothesized by researchers like Van de Wiel (1997) - but also by sample 
composition. Of course, caution is necessary while interpreting the results of the 
present study due to the limited number of females in our data sample. Arguably, the 
ideal way to examine possible gendered sources of recall measures is to conduct a 
longitudinal analysis that examines whether gender differences remain over time. 
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Another way to examine possible gender effects is to include larger sample sizes and 
equal division of female and male subjects across level of expertise. Since gender as 
such doesn’t “explain” any other variable, it is important to take variables into 
account that are as well strongly gender related as theoretically associated to cognitive 
recall. 

The second finding in our study suggests that case processing is also influenced by 
mood. Traditional studies focused on mood changes as related to recall. The present 
study focuses on mood as such. Two findings are surprising. First, it is shown that 
mood as such is moderately correlated with recall. Second, certain mood dimensions 
are associated with different recall functions (literal recall or inferences during recall). 
Both findings seem to refer to cognitive processes that facilitate processing, storage 
and retrieval of cues embedded in the cases. What seems conceptually unexplained is 
why positive mood influences recall, and negative mood is associated with inferences. 
This finding is even more peculiar since only case A reveals this pattern. Although 
these findings seem suggestive, it is important to consider that self-reported positive 
mood may subjects think harder and process the material more deeply when they are 
interested or attentive (Mayer, 1999). At the same time, it remains conceptually 
unclear why negative mood is positively correlated with inferences. It seems that 
more work must be undertaken before a clear evaluation can be made on the 
mechanisms between mood and recall measures. 
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Appendix 



The PANAS 

This scale consists of a number of words that describe different feelings and emotions. Read 
each item and then mark the appropriate answer in the space next to that word. Indicate to 
what extent [INSERT APPROPRIATE TIME INSTRUCTIONS HERE]. Use the following 
scale to record your answers. 



1 


2 


3 


4 


5 


Very slightly 
Or not at all 


a little 


moderately 


quite a bit 


extremely 



Interested 

Distressed 

Excited 

Upset 

Strong 

Guilty 

Scared 

Hostile 

Enthusiastic 

Proud 



Irritable 

Alert 

Ashamed 

Inspired 

Nervous 

Determined 

Attentive 

Jittery 

Active 

Afraid 



We have used PANAS with the following time instructions: 

Moment (you feel this way right now, that is at the present moment) 

Past two weeks (you have felt this way during the past two weeks) 



Cases Used In This Experiment. 



Case A: Flex, Ltd. 

Mr. Fox is director and founder of an employment agency (Flex, Ltd.). More than 25 local 
offices are divided in several regions. Most offices are located in the South of the 
Netherlands. The state of Zeeland covers more offices than the state of Limburg. Flex 
showed a remarkable growth: from 70 to 150 employees, sales growing from NLG 15 million 
to NLG 50 million. Average sales of competitors are about $ 60 million. 

Mr. Fox’s desk contains a growing amount of files. Decision-making is getting more and 
more complex and tiresome. A growing amount of files remain closed. Mr. Fox prefers an 
informal organization where employees can just drop in his office. Local directors contact 
Mr. Fox frequently for detailed issues, meanwhile complaining about a lack of central policy 
making and clear company vision. 

Due to the development of new kinds of jobs. Flex decided to specialize. Different offices 
specialize in different job areas. Offices are specialized in administration, industry, general 
management and health care. This is unlike in the past when offices covered all job areas. It 
was not unusual that even physicians and architects became new jobs through the same office 
of Flex Company. 

Local offices are not used to this change. Directors of these offices complain about their job 
because it is no longer challenging now that they are only targeting at a single profession or a 
single job area. It shows that local offices are often getting in conflict with other local 
offices when negotiating with the same company or client. In addition some local offices 
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initiated their own market research. Clients (companies looking for new personnel) 
complain that they have to contact different offices for different job specialties. 

Mr. Fox considers possibilities to improve the performance of Flex. He thinks about 
splitting company’ shares. The accounting department, however, is convinced that Mr. Fox 
should lower his expenses for public-relations and sponsoring (Mr. Fox considers becoming 
a sponsor for a tennis-club). The accounting department argues that more money should be 
allocated to hiring new people for the local offices, or development of a new house-style for 
the entire organization. The IT-department is convinced that the development of a new 
financial accounting system provides better and faster information about the company’s cash- 
flow. In addition the write-off period for computers should be reduced to four years. 



Case B: CreaLtd. 

Miss Jones is looking irritated through a big window. She is not satisfied about the 
collaborative work at her department. She is now eleven years employed at Crea Ltd. an 
advertising agency. Her department develops creative advertising ideas for the ‘accounts’ 
department. After graduation, Miss Jones started as assistant-manager at the logistics 
department. A few years ago she became head of the creativity department. 

Momentarily she is working on week schedules and setting priorities for her employees. She 
gives assignments to individual employees. These assignments are more of the same for her 
employees. In her view efficiency is increased if individuals get assignnfients they are most 
experienced with. Miss Jones herself handles jobs requiring coordination and responsibility. 
Every week she monitors the work of each individual at her department. In her opinion the 
department is very well managed. 

Until recently her department consisted of five employees who immediately after graduation 
(BA-degree) joined her department. Three months ago new people from another small and 
very young advertising agency joined her group. This agency employed different working 
procedures, mainly based on self-regulated teams. Some of these people received a Master 
degree in History others received Master degrees in communication. Miss Jones is 
dissatisfied with one of the new employees (Mister Williams). First he didn’t want to log his 
work in a journal, now he has taken initiatives she never asked for. Mister Williams claims 
he never experienced problems in the past. But now he simply can not find his way in the 
organization. On behalf of the other new employees he stated that the creativity department 
would be better off without Miss Jones. 

Miss Jones feels insecure about what is happening and considers developing clear-cut 
working schedules for Mr. Williams. Otherwise she is afraid he may disturb routine 
procedures in her department. Last month she notified the Director of the company about the 
case of Mr. Williams. Unfortunately, this week was her first opportunity to discuss it with 
the director (he was on a business trip to Russia to examine possibilities for partnerships). 
She claims that the old group of employees didn’t have any problems with her. But the 
director does not agree and introduces the subject of low productivity within her department, 
and the fact that people left her department. In addition he points to small over-expenditures 
in her operational budget. According to Miss Jones this may be explained by higher costs for 
the yearly party with her department. The director (showing a resolute stand) says that Mr. 
Williams should get a fair chance because he developed some very interesting advertising 
campaigns in the past. That is the reason why the director hired new people for Miss Jones’ 
department. 
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